Planning and Reporting Effective Web-Based RAND/UCLA Appropriateness Method Panels: Literature Review and Preliminary Recommendations.

BACKGROUND: The RAND/UCLA Appropriateness Method (RAM), a variant of the Delphi Method, was developed to synthesize existing evidence and elicit the clinical judgement of medical experts on the appropriate treatment of specific clinical presentations. Technological advances now allow researchers to conduct expert panels on the internet, offering a cost-effective and convenient alternative to the traditional RAM. For example, the Department of Veterans Affairs recently used a web-based RAM to validate clinical recommendations for de-intensifying routine primary care services. A substantial literature describes and tests various aspects of the traditional RAM in health research; yet we know comparatively less about how researchers implement web-based expert panels. OBJECTIVE: The objectives of this study are twofold: (1) to understand how the web-based RAM process is currently used and reported in health research and (2) to provide preliminary reporting guidance for researchers to improve the transparency and reproducibility of reporting practices. METHODS: The PubMed database was searched to identify studies published between 2009 and 2019 that used a web-based RAM to measure the appropriateness of medical care. Methodological data from each article were abstracted. The following categories were assessed: composition and characteristics of the web-based expert panels, characteristics of panel procedures, results, and panel satisfaction and engagement. RESULTS: Of the 12 studies meeting the eligibility criteria and reviewed, only 42% (5/12) implemented the full RAM process with the remaining studies opting for a partial approach. Among those studies reporting, the median number of participants at first rating was 42. While 92% (11/12) of studies involved clinicians, 50% (6/12) involved multiple stakeholder types. Our review revealed that the studies failed to report on critical aspects of the RAM process. For example, no studies reported response rates with the denominator of previous rounds, 42% (5/12) did not provide panelists with feedback between rating periods, 50% (6/12) either did not have or did not report on the panel discussion period, and 25% (3/12) did not report on quality measures to assess aspects of the panel process (eg, satisfaction with the process). CONCLUSIONS: Conducting web-based RAM panels will continue to be an appealing option for researchers seeking a safe, efficient, and democratic process of expert agreement. Our literature review uncovered inconsistent reporting frameworks and insufficient detail to evaluate study outcomes. We provide preliminary recommendations for reporting that are both timely and important for producing replicable, high-quality findings. The need for reporting standards is especially critical given that more people may prefer to participate in web-based rather than in-person panels due to the ongoing COVID-19 pandemic.

Diverse Recruitment Strategies Are Needed to Reduce Digital Divide: Results from a Workshop Addressing Digital Divide and Effects of Pandemic Restrictions.

Recruitment is a bottleneck for research - especially digital health studies. Studies often focus on those who are easy to reach or already engaged in their health, leaving those who are uninterested or un-engaged, as "un-reached". This contributes to the "digital divide". COVID-19 restrictions made recruitment more difficult. During a virtual workshop of our peers, we discussed recruitment of un-reached groups for digital health studies, especially during COVID-times. All agreed; we need to go where the un-reached are by collaborating with community-based services and organizations.

Clinical trials in COVID-19 management & prevention: A meta-epidemiological study examining methodological quality.

OBJECTIVE: To describe the characteristics of Covid-19 randomized clinical trials (RCTs) and examine the association between trial characteristics and the likelihood of finding a significant effect. STUDY DESIGN: We conducted a systematic review to identify RCTs (up to October 21, 2020) evaluating drugs or blood products to treat or prevent Covid-19. We extracted trial characteristics (number of centers, funding sources, and sample size) and assessed risk of bias (RoB) using the Cochrane RoB 2.0 tool. We performed logistic regressions to evaluate the association between RoB due to randomization, single vs. multicentre, funding source, and sample size, and finding a statistically significant effect. RESULTS: We included 91 RCTs (n = 46,802); 40 (44%) were single-center, 23 (25.3%) enrolled <50 patients, 28 (30.8%) received industry funding, and 75 (82.4%) had high or probably high RoB. Thirty-eight trials (41.8%) reported a statistically significant effect. RoB due to randomization and being a single-center trial were associated with increased odds of finding a statistically significant effect. CONCLUSIONS: There is high variability in RoB among Covid-19 trials. Researchers, funders, and knowledge-users should be cognizant of the impact of RoB due to randomization and single-center trial status in designing, evaluating, and interpreting the results of RCTs. REGISTRATION: CRD42020192095.

A new standard scoring for interstitial pneumonia based on quantitative analysis of ultrasonographic data: A study on COVID-19 patients.

OBJECTIVE: To assess the effectiveness of 3 novel lung ultrasound (LUS)-based parameters: Pneumonia Score and Lung Staging for pneumonia staging and COVID Index, indicating the probability of SARS-CoV-2 infection. METHODS: Adult patients admitted to the emergency department with symptoms potentially related to pneumonia, healthy volunteers and clinical cases from online accessible databases were evaluated. The patients underwent a clinical-epidemiological questionnaire and a LUS acquisition, following a 14-zone protocol. For each zone, a Pneumonia score from 0 to 4 was assigned by the algorithm and by an expert operator (kept blind with respect to the algorithm results) on the basis of the identified imaging signs and the patient Lung Staging was derived as the highest observed score. The output of the operator was considered as the ground truth. The algorithm calculated also the COVID Index by combining the automatically identified LUS markers with the questionnaire answers and compared with the nasopharyngeal swab results. RESULTS: Overall, 556 patients were analysed. A high agreement between the algorithm assignments and the expert operator evaluations was observed, both for Pneumonia Score and Lung Staging, with the latter having sensitivity and specificity over 92% both in the discrimination between healthy/sick patients and between sick patients with mild/severe pneumonia. Regarding the COVID Index, an area under the curve of 0.826 was observed for the classification of patients with/without SARS-CoV-2. CONCLUSION: The proposed methodology allowed the identification and staging of patients suffering from pneumonia with high accuracy. Moreover, it provided the probability of being infected by SARS-CoV-2.

Building Equitable Patient Partnerships during the COVID-19 Pandemic: Challenges and Key Considerations for Research and Policy.

The unequal social and economic burden of the COVID-19 pandemic is evident in racialized and low-income communities across Canada. Importantly, social inequities have not been adequately addressed and current public policies are not reflective of the needs of diverse populations. Public participation in decision-making is crucial and there is, therefore, a pressing need to increase diversity of representation in patient partnerships in order to prevent the further exclusion of socially marginalized groups from research and policy making. Deliberate effort and affirmative action are needed to meaningfully engage and nurture diverse patient partnerships by broadening the scope of the patient community to include excluded or underrepresented individuals or groups. This will help us co-develop ways to enhance access and equity in healthcare and prevent the systematic reproduction of structural inequalities that have already been heightened by the COVID-19 pandemic.

Para sport translation of the IOC consensus on recording and reporting of data for injury and illness in sport.

In 2020, the IOC proposed a universal methodology for the recording and reporting of data for injury and illness in sport. Para sport is played by individuals with impairment, and they have a unique set of considerations not captured by these recommendations. Therefore, the aim of this addendum to IOC consensus statement was to guide the Para sport researcher through the complexities and nuances that should be taken into consideration when collecting, registering, reporting and interpreting data regarding Para athlete health. To develop this translation, experts in the field of Para sports medicine and epidemiology conducted a formal consensus development process, which began in March 2020 with the formation of a consensus group that worked over eight phases, incorporating three virtual consensus meetings to finalise the translation. This translation is consistent with the IOC consensus statement, yet provides more detailed Para athlete specific definitions and recommendations on study population, specifically, diagnostic and eligible impairment categorisation and recording of adaptive equipment, and defining and classifying health problems in the context of Para sport. Additionally, recommendations and Para athlete specific examples are described with regards to injury mechanism, mode of onset, injury and illness classification, duration, capturing and reporting exposure and risk. Finally, methods and considerations are provided to cater to the varied needs of athletes with impairment with respect to data collection tools. This harmonisation will allow the science to develop and facilitate a more accurate understanding of injury and illness patterns for tailoring evidence-informed prevention programmes and enabling better planning of medical services for Para sport events.

Reporting and methodological quality of COVID-19 systematic reviews needs to be improved: an evidence mapping.

OBJECTIVES: To assess the reporting and methodological quality of COVID-19 systematic reviews, and to analyze trends and gaps in the quality, clinical topics, author countries, and populations of the reviews using an evidence mapping approach. STUDY DESIGN AND SETTING: A structured search for systematic reviews concerning COVID-19 was performed using PubMed, Embase, Cochrane Library, Campbell Library, Web of Science, CBM, WanFang Data, CNKI, and CQVIP from inception until June 2020. The quality of each review was assessed using the Assessment of Multiple Systematic Reviews 2 (AMSTAR 2) checklist and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist. RESULTS: In total, 243 systematic reviews met the inclusion criteria, over 50% of which (128, 52.7%) were from 14 developing countries, with China contributing the most reviews (76, 31.3%). In terms of methodological quality of the studies, 30 (12.3%) were of moderate quality, 63 (25.9%) were of low quality, and 150 (61.7%) were of critically low quality. In terms of reporting quality, the median (interquartile range) PRISMA score was 14 (10-18). Regarding the topics of the reviews, 24 (9.9%) focused on the prevalence of COVID-19, 69 (28.4%) focused on the clinical manifestations, 30 (12.3%) focused on etiology, 43 (17.7%) focused on diagnosis, 65 (26.7%) focused on treatment, 104 (42.8%) focused on prognosis, and 25 (10.3%) focused on prevention. These studies mainly focused on general patients with COVID-19 (161, 66.3%), followed by children (22, 9.1%) and pregnant patients (18, 7.4%). CONCLUSION: This study systematically evaluated the methodological and reporting quality of systematic reviews of COVID-19, summarizing and analyzing trends in their clinical topics, author countries, and study populations.

ROSES-S: Statement from the World Health Organization on the reporting of seroepidemiologic studies for SARS-CoV-2.

Well-designed population-based seroepidemiologic studies can be used to refine estimates of infection severity and transmission, and are therefore an important component of epidemic surveillance. However, the interpretation of the results of seroepidemiologic studies for SARS-CoV-2 has been hampered to date principally by heterogeneity in the quality of the reporting of the results of the study and a lack of standardized methods and reporting. We provide here the ROSES-S: Reporting of Seroepidemiologic studies-SARS-CoV-2. This is an updated checklist of 22 items that should be included in the reporting of all SARS-CoV-2 seroepidemiologic studies, irrespective of study design.

A Productive Scholar's Guide to Respectful, Responsible Inquiry During the COVID-19 Pandemic: Moving Forward.

As a result of the COVID-19 pandemic of 2020, schools across the country have pivoted to providing a range of instructional opportunities including remote, hybrid, and modified in-person options with a commitment to keeping our society safe and supporting our nations' youth in continuing their education. One byproduct of this most challenging situation has been the interruption of scientific inquiry in many fields, including educational research. Using a framework of self-leadership, in this feature article, we provide reflections to guide discussion among scholars involved in school-based research during and after the COVID-19 pandemic. Based on lessons learned in our own mentee-mentor relationships and from our collective experiences, the purpose of this article is to offer considerations primarily for early-career scholars, but also to engage dialogue among mid- and late-career scholars to promote school-based inquiry in this defining moment in history, at a time when educators are concerned with mitigating learning loss. We focus on (a) revisiting and refining programmatic lines of inquiry, (b) crafting articles to disseminate lessons learned from research conducted to date-including those with lost time points, (c) retooling in a focused area of inquiry (e.g., methodology), (d) conceptualizing future studies featuring procedures that do-and do not-allow for interpersonal contact, and (e) appreciating and acknowledging individuals who have facilitated inquiry. We conclude with a note of encouragement and call to action for future educational research.

Methodological Analysis: Randomized Controlled Trials for Pfizer and Moderna COVID-19 Vaccines.

Critical appraisal of randomized controlled trials (RCTs) determines rigor, quality, and whether the findings are applicable to the populations served in clinical practices. The authors conducted a rigorous analysis using the RCT Critical Appraisal Skills Programme (CASP) Checklist for the two RCTs Pfizer (New York, NY) and Moderna (Cambridge, MA) conducted and the reporting of these RCTs using the Consolidated Standards of Reporting Trials checklist. The goals for this analysis were twofold: (1) enable health care providers to understand the methods and outcomes of these RCTs, and (2) enable health care providers and community leaders to become champions for the vaccines to reduce vaccine hesitancy among all populations. The analysis is presented using each of the 11 questions on the CASP tool while comparing the methodology and results for each vaccine. Most CASP tool items were positive or yes for both the Pfizer and Moderna RCTs. Items that were not scored as yes are discussed. The analysis outcomes revealed that both RCTs were rigorously conducted and provide an assurance to all health care providers and the public of the safety and efficacy of both vaccines to impact the astounding morbidity and mortality of COVID-19 disease. The authors believed that the analysis was an essential component of the distribution process to develop plans and communication strategies to reduce potential vaccine hesitancy and resistance.

Methodology over metrics: current scientific standards are a disservice to patients and society.

Covid-19 research made it painfully clear that the scandal of poor medical research, as denounced by Altman in 1994, persists today. The overall quality of medical research remains poor, despite longstanding criticisms. The problems are well known, but the research community fails to properly address them. We suggest that most problems stem from an underlying paradox: although methodology is undeniably the backbone of high-quality and responsible research, science consistently undervalues methodology. The focus remains more on the destination (research claims and metrics) than on the journey. Notwithstanding, research should serve society more than the reputation of those involved. While we notice that many initiatives are being established to improve components of the research cycle, these initiatives are too disjointed. The overall system is monolithic and slow to adapt. We assert that top-down action is needed from journals, universities, funders and governments to break the cycle and put methodology first. These actions should involve the widespread adoption of registered reports, balanced research funding between innovative, incremental and methodological research projects, full recognition and demystification of peer review, improved methodological review of reports, adherence to reporting guidelines, and investment in methodological education and research. Currently, the scientific enterprise is doing a major disservice to patients and society.

Challenges and Lessons Learned From COVID-19 Trials: Should We Be Doing Clinical Trials Differently?

The COVID-19 crisis led to a flurry of clinical trials activity. The COVID-evidence database shows 2814 COVID-19 randomized trials registered as of February 16, 2021. Most were small (only 18% have a planned sample size > 500) and the rare completed ones have not provided published results promptly (only 283 trial publications as of February 2021). Small randomized trials and observational, nonrandomized analyses have not had a successful track record and have generated misleading expectations. Different large trials on the same intervention have generally been far more efficient in producing timely and consistent evidence. The rapid generation of evidence and accelerated dissemination of results have led to new challenges for systematic reviews and meta-analyses (eg, rapid, living, and scoping reviews). Pressure to regulatory agencies has also mounted with massive emergency authorizations, but some of them have had to be revoked. Pandemic circumstances have disrupted the way trials are conducted; therefore, new methods have been developed and adopted more widely to facilitate recruitment, consent, and overall trial conduct. On the basis of the COVID-19 experience and its challenges, planning of several large, efficient trials, and wider use of adaptive designs might change the future of clinical research. Pragmatism, integration in clinical care, efficient administration, promotion of collaborative structures, and enhanced integration of existing data and facilities might be several of the legacies of COVID-19 on future randomized trials.